Remote coordinate input device and remote coordinate input method

ABSTRACT

The invention prevents an audience from being distracted by movements of a demonstrator that are not related to movements of a pointer on a screen, and by the demonstrator moving away from the screen, during a presentation which is performed by enlarging and projecting the display of a personal computer onto the screen by a projector. A designating tool, which is held by the hand of the demonstrator, is imaged by an imaging part which is disposed at the top of the screen. On the front of the designating tool, infrared LEDs are disposed at each vertex of an isosceles triangle, and recessed at the center, and the orientation of the designating tool is obtained from the positional relationship of the infrared LEDs of an image which has been imaged by the imaging part. This orientation is converted to planar coordinates and is sent to a computer, a marker is displayed on a screen, and the software is operated.

This application is a Division of Application Ser. No. 09/188,146 filedNov. 9, 1998, now U.S. Pat. No. 6,317,118.

BACKGROUND OF THE INVENTION

1. Field of Invention

The present invention relates to a remote coordinate input device andinput method used when either designating a specified portion on ascreen, or operating application software when performing a presentationby projecting an enlarged display of a computer screen onto a screenusing a projector.

2. Description of Related Art

In a conventional presentation, information is displayed on a largescreen by projecting an enlarged display of a transparent-type documentusing an OHP (overhead projector), or by projecting an enlarged filmimage using a slide projector. A demonstrator conducts the presentationwhile designating important points of the projected image using apointing stick or a laser pointer.

Moreover, in recent years, due to the wide distribution of personalcomputers and liquid crystal projectors, effective presentations havebeen performed using various application software that have beendeveloped for presentations.

However, a mouse or similar device which is connected to the computermust be operated in order to display the screen. In order to operate themouse or similar device, the demonstrator must move away from being infront of the screen. Unfortunately, this movement by the demonstratorand the movement of the pointer on the screen are not related to eachother. Thus, the audience is sometimes distracted.

In order to avoid this problem, devices which do not restrict theposition of the demonstrator when the demonstrator is operating thecomputer, and which allow the operator to operate the applicationsoftware by directly operating on the screen from a comparatively freeposition, and which can input so-called remote coordinates are beingdeveloped, and various corresponding methods have also been developed.

As one example of these methods, Japanese Patent Laid-Open PublicationNo. Hei 5-19953 discloses a method of controlling displayed informationby operating a remote control device from a position which is distantfrom the screen. However, the operation is only performed by variouskinds of keys that are provided on the remote control device. Forexample, in order to move the pointer on the screen, arrow keys have tobe pressed. However, in this method, the pointer only movesintegratedly, which provides a different feel than one gets frompointing by using a pointing stick.

Moreover, Japanese Patent Laid-Open Publication No. Hei 2-300816discloses a method in which three light emitting elements are providedon a designating part. Each light emitting element radiates light in adifferent direction. The angle of orientation of the designating part isdetected by the difference in strength of the light received by thelight receiving elements from the light radiated from the light emittingelements. However, since the light has to be modulated in order todistinguish the light emitting elements, the apparatus is subject toproblems, such as the designating part becoming complicated and it beingeasily influenced by external noise because the device uses the lightamount difference.

Moreover, both of these methods have difficulty in distinguishingbetween two points or more that are simultaneously designated by thedemonstrator and another person, such as someone in the audience.

SUMMARY OF THE INVENTION

It is therefore an object of the invention to solve the above-mentionedproblems. The remote coordinate input device in accordance with theinvention includes designating means that has at least three first lightemitting elements and a second light emitting element that is arrangedon an orthogonal axis that is perpendicular to a plane delineated bysaid first light emitting elements, an imaging means that images arelative positional relationship of said first and second light emittingelements of said designating means, a coordinate converting means thatobtains a direction in which said designating means points with respectto said imaging means from a picture signal which is imaged by saidimaging means and converts it to a planar coordinate, an output meansthat outputs the coordinate that is obtained by said coordinateconverting means, and a display means that displays designatinginformation on a screen based on said coordinate that is obtained bysaid output means.

In accordance with another aspect of the remote coordinate input device,said first light emitting elements are respectively arranged at eachvertex of an isosceles triangle, and a base of the triangle whichconnects two of the first light emitting elements is substantiallyhorizontally arranged.

In accordance with another aspect of the remote coordinate input device,said first light emitting elements are respectively arranged at eachvertex of a rhombus, and one diagonal line of said rhombus issubstantially horizontally arranged.

Another aspect of the remote coordinate input device further includes amodulating means that modulates said first or second light emittingelements according to operating information from an operating means thatis provided on said designating means, and a light receiving means thatdetects the modulated operating information.

Another aspect of the remote coordinate input device further includes ahand detecting means that detects whether said designating means isbeing held by a hand of the user, and controls lighting or turning offof the light of said first and second light emitting elements by anoutput of said hand detecting means.

Another remote coordinate input device in accordance with the inventionincludes a designating means that has at least three first reflectingelements and a second reflecting element that is arranged on anorthogonal axis which is perpendicular to a plane delineated by saidfirst reflecting elements, an irradiating means that irradiates saidfirst and second reflecting elements of said designating means, animaging means that images a relative positional relationship of saidfirst and second reflecting elements of said designating means, acoordinate converting means that obtains a direction in which saiddesignating means points with respect to said imaging means from apicture signal that has been imaged by said imaging means, and convertsit into a planar coordinate, an output means that outputs the coordinatethat is obtained by said coordinate converting means, and a displaymeans that displays designating information on a screen based on saidcoordinate that is obtained from said output means.

In accordance with another aspect of this remote coordinate inputdevice, said first reflecting elements are respectively arranged at eachvertex of an isosceles triangle, and a base of the triangle whichconnects two of the reflecting elements is substantially horizontallyarranged.

In accordance with another aspect of this remote coordinate inputdevice, said first reflecting elements are respectively arranged at eachvertex of a rhombus and one diagonal line of said rhombus issubstantially horizontally arranged.

Another remote coordinate input device in accordance with the inventionincludes a designating means that has a hollow disk-shaped firstreflecting element and a second reflecting element which is arranged onan orthogonal axis that is perpendicular to a plane delineated by saidfirst reflecting element, an irradiating means that irradiates saidfirst and second reflecting elements of said designating means, animaging means that images a relative positional relationship of saidfirst and second reflecting elements of said designating means, acoordinate converting means that obtains a direction in which saiddesignating means points with respect to said imaging means from asignal which has been imaged by said imaging means and converts it to aplanar coordinate, an output means that outputs the coordinate that isobtained by said coordinate converting means, and a display means thatdisplays designating information on a screen based on said coordinatethat is obtained by said output means.

A remote coordinate input method in accordance with the inventionincludes the steps of obtaining a first image with a first lightemitting element or a first reflecting element, obtaining a second imagewith a said second light emitting element or a second reflectingelement, obtaining a reference coordinate from a coordinate of saidfirst image and a direction with respect to an imaging means of adesignating means from the positional relationship of said second imageand said reference coordinate, and specifying a designating position ina display means according to said direction.

Another aspect of the remote coordinate input method further includesobtaining independent designating tool images in which an image of aplurality of designating means is separated into independent images fromthe image which has been imaged by said imaging means, obtaining adirection with respect to said imaging means of said designating meansfor every independent designating tool image, and specifying adesignated position by said display means according to said directions.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing a first embodiment of the present invention.

FIG. 2 is a diagram showing one example of a designating tool of thefirst embodiment of the present invention.

FIG. 3 is a three-face-view explaining the details of a front part ofthe designating tool of the first embodiment of the present invention.

FIG. 4 is a diagram showing one example of an imaging part of the firstembodiment of the present invention.

FIG. 5 is a diagram showing one example of an output image of a CCDcamera of the first embodiment and a second embodiment of the presentinvention.

FIG. 6 is a plan view showing the positions of LEDs of the designatingtool of the first embodiment of the present invention.

FIG. 7 is a side view showing positions of the LEDs of the designatingtool of the first embodiment of the present invention.

FIG. 8 is a flow chart showing a procedure of obtaining the coordinatesin accordance with the first and second embodiments of the presentinvention.

FIG. 9 is a diagram showing another example of a designating tool of thefirst embodiment of the present invention.

FIG. 10 is a three-face-view showing details of the front part of thedesignating tool of the first embodiment of the present invention.

FIG. 11 is a diagram showing one example of the output image of the CCDcamera of the first embodiment of the present invention.

FIG. 12 is a plan view showing positions of the LEDs of the designatingtool of the first embodiment of the present invention.

FIG. 13 is a side view showing positions of the LEDs of the designatingtool of the first embodiment of the present invention.

FIG. 14 is a diagram showing the second embodiment of the presentinvention.

FIG. 15 is a diagram showing one example of a designating tool of thesecond embodiment of the present invention.

FIG. 16 is a three-face-view showing details of the front part of thedesignating tool of the second embodiment of the present invention.

FIG. 17 is diagram showing one example of an irradiating part and animaging part of the second embodiment of the present invention.

FIG. 18 is a plan view showing the position of the reflecting members ofthe designating tool of the second embodiment of the present invention.

FIG. 19 is a side view showing the position of the reflecting members ofthe designating tool of the second embodiment of the present invention.

FIG. 20 is a diagram showing a third embodiment of the presentinvention.

FIG. 21 is a diagram showing one example of a designating tool of thethird embodiment of the present invention.

FIG. 22 is a three-face-view showing details of the front part of thedesignating tool of the third embodiment of the present invention.

FIG. 23 is a diagram showing one example of an output image of a CCDcamera of the third embodiment of the present invention.

FIG. 24 is a plan view showing the position of the reflecting members ofthe designating tool of the third embodiment of the present invention.

FIG. 25 is a side view showing the position of the reflecting members ofthe designating tool of the third embodiment of the present invention.

FIG. 26 is a flow chart showing a procedure of obtaining coordinates ofthe third embodiment of the present invention.

FIG. 27 is a flow chart showing a procedure which divides a plurality ofimages of the designating tool of the first, second, and thirdembodiments of the present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The first embodiment of the present invention is explained based on thedrawings.

FIG. 1 shows a first embodiment of the present invention. Display dataof a computer 10 is sent to a projector 4, projected, enlarged, anddisplayed on a screen 7.

An imaging part 5 is provided at the top of the screen 7. The imagingpart 5 images a designating tool 1, which is held by a demonstrator 9,detects the orientation of the designating tool 1, converts it intocoordinates and transfers it to the computer 10. A receiver 11 isprovided at the bottom of the screen 7, which detects a specified signalfrom the designating tool 1 and transfers it to the computer 10.

During a presentation, the position of the marker 12 can be shifted bychanging the orientation of the designating tool while the demonstrator9 is watching the screen.

The marker 12 is the same as a pointer or a cursor that is operated by aconventional mouse. When the orientation of the designating tool 1 ischanged, the marker 12 is shifted, which is similar to the movement of apointer or cursor when the mouse is shifted on its corresponding plane.

One example of the designating tool 1 is shown in FIG. 2. A plurality ofinfrared LEDs (light emitting diodes) 21 that irradiate infrared raysare provided on a front face. Operating buttons 22 are provided on a topsurface. Hand detecting electrodes 23 are provided on a portion which isheld by a hand of the demonstrator. Moreover, batteries, which are notshown in the figure, are housed inside the designating tool 1.

FIG. 3 is a three-face-view which shows details of the front face of thedesignating tool 1. The infrared LEDs 21A, 21B and 21C, which are thefirst light emitting elements, are disposed in the same plane atvertices of an isosceles triangle. The infrared LEDs 21A and 21B areseparated by a distance “a”. The line connecting LEDs 21A and 21B isparallel to the top surface of the designating tool 1. The infrared LED21C is separated by a distance “b” in the vertical direction from thecenter of the line which connects LEDs 21A and 21B. An infrared LED 21E,which is the second light emitting element, is separated by a distance“b/2” in the vertical direction from 21C in the front face, and isdisposed at a position which is recessed by a distance “d” inside thedesignating tool 1. It is not necessary for LED 21 E to be disposed at aposition on a line which is perpendicular to the plane which isdelineated by LEDs 21A, 21B and 21C. However, compactness of the devicecan be obtained by arranging it as shown in FIG. 3.

When the demonstrator 9 holds the designating tool 1, a hand detectingelectrode 23 detects the fact that the demonstrator 9 is holding thedesignating tool 1, lights the infrared LEDs 21 (21A, 21B, 21C and 21E),and turns the lights off when the demonstrator 9 removes his or herhand. Moreover, when one of the operating buttons 22 is pressed, theinfrared LEDs 21 are all turned off at once, and the information of theoperating button 22 that is pressed is transmitted via timed flashesthat correspond to a predetermined code of the operating button 22. Thetransmitted code is received by the receiver 11 and is sent to thecomputer 10. When the transmission is completed, all the LEDs are litagain. Moreover, it is possible to assign functions to the operatingbuttons 22, such as a function that corresponds to click button of anormal mouse, and a function which corresponds to application softwarewhich is used in the computer 10.

Next, one example of the imaging part 5 is shown in FIG. 4. A lens 32and an infrared ray filter 33 are provided in a CCD camera 31, and animage of the infrared LED 21 of the designating tool 1 is imaged. Theoutput of the CCD camera 31 is connected to an image processor 34. Theimage processor 34 calculates the planar coordinates on the screen 7based on the image of LED 21 of the designating tool 1 which has beenimaged by the CCD camera 31, and sends the planar coordinates to thecomputer 10 via an output part 35.

One example of the output image of the CCD camera 31 is shown in FIG. 5.Since the infrared ray filter 33 is disposed in front of the lens 32 ofthe CCD camera 31, light of interior illuminations or the like issubstantially entirely removed, and only the images 41 from the LEDs 21are output, as shown in FIG. 5.

Here, the appearance of images 41 from the designating tool 1 isexplained with reference to the figures.

FIG. 6 is a plan view in which the LEDs 21 of the designating tool 1 areviewed from the top part. The center line 26 connects a midpoint betweenthe infrared LEDs 21A and 21B, and the lens 32. FIG. 6 shows the casewhen the designating tool 1 faces a left diagonal direction by an anglel from the center line 26. When the projection plane 24 is assumed alonglines extending perpendicularly from the center line 26, the spacebetween the infrared LEDs 21A and 21B becomes “x” in the projectionplane 24, and the infrared LED 21E is projected as being shifted by “h”from the center line to the left.

FIG. 7 is a side view of the LEDs 21 of the designating tool 1. Thecenter line 26 connects a midpoint between the infrarad LEDs 21A, 21Band 21C and the lens 32. FIG. 7 shows a case when the designating tool 1faces a downward diagonal direction at an angle “m” from center line 26.When the projection plane 24 is assumed along lines extendingperpendicularly from the center line 26, in the projection plane 24, thespace between the infrared LEDs 21A, 21B and 21C becomes “y”, and theinfrared LED 21E is projected as being shifted upward by a distance “v”from the center line.

Here, the images 41 which have been imaged by the CCD camera 31 can beconsidered as images which have been imaged at an arbitrarymagnification at the projection plane 24 of FIGS. 6 and 7. Therefore,the images 41 of FIG. 5 have the same geometrical relationships as theprojected images in the projection plane 24. In FIG. 5, images 41A, 41B,41C and 41E are the images of infrared LEDs 21A, 21B, 21C and 21E,respectively. Moreover, the distance between the images 41A and 41B is“X”, the distance from the center of a line which connects 41A and 41Bto the image 41C in the vertical direction is “Y”, the position which isa distance Y/2 above the image 41C is a reference point 42, thehorizontal component of the distance between the image 41E and thereference point 42 is H, and the vertical component of the distancebetween the image 41E and the reference point 42 is V. Since thereference point 42 lies on an extension of the center line 26, eachvalue of x, h, y and v of FIGS. 6 and 7 becomes proportional to therespective value of X, H, Y or V of FIG. 5. Accordingly, when therelationships between images 41A, 41B, 41C and 41E of the images 41 arechecked, it is possible to find out how much the designating tool 1 isinclined horizontally and vertically with respect to the lens 32.

[Equation 1]

x=a cos l

h=d sin l${\therefore\frac{h}{x}} = {{{\frac{d}{a}\tan \quad l}\therefore l} = {\tan^{- 1}\left( {\frac{a}{d}\quad \frac{h}{x}} \right)}}$

[Equation 2]

y=b cos m

v=d sin m${\therefore\frac{v}{y}} = {{{\frac{d}{b}\tan \quad m}\therefore\quad m} = {\tan^{- 1}\left( {\frac{b}{d}\quad \frac{v}{y}} \right)}}$

Equation 1 is an equation which shows the relationship of a horizontaldirection of the projection plane 24. Moreover, equation 2 is anequation which shows the relationship of a vertical direction of theprojection plane 24. As described above, each value of x, h, y and is vis proportional to each respective value of X, H, Y and V. ThereforeEquations 1 and 2 can be described as Equation 3 as follows:

[Equation 3]$l = {\tan^{- 1}\left( {\frac{a}{d}\quad \frac{H}{X}} \right)}$$m = {\tan^{- 1}\left( {\frac{b}{d}\quad \frac{V}{Y}} \right)}$

Here, since the values of a, b and d are already known values of thedesignating tool 1, the angles l and m can be obtained from the images41 of FIG. 5.

Moreover, the data which is output to the computer 10 from the outputpart 35 provides horizontal and vertical orthogonal coordinates of theplane which is projected onto the screen 7. Therefore, when the centerof the screen is the point of origin, as shown in Equation 4, the anglesl and m can be converted into the coordinates Xs, Ys.

[Equation 4] $\begin{matrix}{{Xs} = {K\quad \tan \quad l}} \\{= {K\quad \frac{a}{d}\quad \frac{H}{X}}}\end{matrix}$ $\begin{matrix}{{Ys} = {K\quad \tan \quad m}} \\{= {K\quad \frac{b}{d}\quad \frac{V}{Y}}}\end{matrix}$

Here, K of Equation 4 is a proportional constant and is a value todetermine an inclination of the designating tool 1 and the sensitivityof the output. This value can be fixed as an appropriate value which iseasy to use, or can be set corresponding to the preference of thedemonstrator 9. Moreover, as demonstrated by Equation 4, the values ofthe angles l and m do not need to be obtained in the actual calculation.

Next, according to the above-mentioned principles, in the imageprocessing device, a method of obtaining coordinates on the screen whichare designated by the designating tool 1 from the image 41 imaged by theCCD camera 31 is explained with reference to the flowchart of FIG. 8.

In S1, the center of balance coordinate of the images 41A, 41B, 41C and41E is obtained. This is to determine an approximately centered positionas a representative point since the image has a limited size, and it canbe obtained by a commonly known calculation method. More accuratecoordinates can be calculated when the center of balance coordinates aredetermined when considering the difference of brightness due to thedistance of each LED from the imaging part being slightly different.

In S2, the distance X of the horizontal direction and an intermediatecoordinate between 41A and 41B are obtained from the center of balancecoordinates of images 41A and 41B.

In S3, the distance Y of the vertical direction and the coordinate ofthe reference point 42, which is the midpoint of both the distance X andthe distance Y, are obtained from the coordinates which were obtained inS2 and the center of balance coordinates of the image 41C.

In S4, the center position of image 41E, and the horizontal distance Hand the vertical distance V of the reference point 42 are obtained.

In S5, the coordinates Xs and Ys on the screen 7 are obtained byEquation 4.

In summary, a method of obtaining the above-mentioned coordinatesincludes the steps of obtaining the distances X and Y and the referencepoint 42 from the images 41A, 41B and 41C of the infrared LEDs 21A, 21Band 21C, which are the first light emitting elements of the designatingtool 1, obtaining horizontal and vertical distances H and V of thereference point 42 from the image 41E of the infrared LED 21E, which isthe second light emitting element, and obtaining the coordinates Xs andYs on the screen 7 by a calculation.

Moreover, here, the center position is the point of origin of thecoordinates on the screen. However, it is possible to make a peripheralpart of the screen the point of origin by providing a bias value.

The case of using only one designating tool 1 is explained above.However, a method of obtaining the coordinates on the screen designatedby respective designating tools when a plurality of designating toolsare simultaneously used is explained below with reference to theflowchart of FIG. 27.

In S1, the image is enlarged and processed for each image. It isacceptable to perform processing to fill in the periphery of the portionin which the image exists with the same signal in accordance with apredetermined size.

Next, in S2, the area of the image for each designating tool is obtainedby performing segmentation.

Since an area of the image for every designating tool is obtained asstated above, it is acceptable to obtain a coordinate in the area foreach image with reference to the flowchart which is shown in FIG. 8.

Next, another example of the designating tool 1 is shown in FIG. 9.

A plurality of infrared LEDs (light emitting diodes) 51 that irradiateinfrared rays are provided in the front face of the designating tool.Operating buttons 52 are provided on the top surface of the designatingtool. A hand detecting switch 53 is provided on the portion which isheld by the demonstrator's hand. Moreover, batteries, which are notshown in the figure, are housed inside the designating tool.

FIG. 10 is a three-face-view which shows the details of the front faceportion of the designating tool. The infrared LEDs 51A, 51B, 51C and51D, which are the first light emitting elements, are disposed in arhombus in the same plane. The infrared LEDs 51B and 51D are separatedby a distance “a”. A line that connects LEDs 51B and 51D is parallel tothe top surface at the demonstrator's hand side of the designating tool1. Infrared LEDs 51A and 51C are separated by a distance “b” in thevertical direction. The infrared LED 51E, which is the second lightemitting element, is disposed at a position which is recessed inside thedesignating tool housing by distance “d” at the intersection of astraight line which connects LEDs 51B and 51C and a straight line whichconnects LEDs 51A and 51C in the front view.

When the demonstrator 9 holds the designating tool 1, the hand detectingswitch 53 detects the fact that the demonstrator 9 is holding thedesignating tool in his or her hand, lights the infrared LEDs 51 (51A,51B, 51C, 51D and 51E), and turns them off when the demonstrator 9removes his or her hand from the designating tool. Moreover, when theoperating button 52 is pressed, the infrared LEDs 51 are all turned offat once, and information of operation button 52 that is pressed istransmitted via timed flashes that correspond to a predetermined code ofthe operation button 52. The transmitted code is received by thereceiver 11 and is sent to the computer 10. When the transmission iscompleted, all the infrared LEDs are lit again. Moreover, the operationbuttons 52 may be assigned the function of a click button of a normalmouse, a function which corresponds to application software which isused in the computer 10, or another function.

Here, the imaging of images 61 that are transmitted by the designatingtool 1 is explained with reference to the figures.

FIG. 12 is a plan view in which the LEDs 51 of the designating tool areviewed from the top. The center line 56 is a straight line whichconnects a midpoint between the infrared LEDs 51B and 51D, and the lens32. FIG. 12 shows the case when designating tool 1 faces a left diagonaldirection at an angle l from the center line 56. When the projectionplane 54 is assumed along lines extending perpendicularly from thecenter line 56, the space between the infrared LEDs 51B and 51D becomes“x” in the projection plane 54, and the infrared LED 51E is projected asbeing shifted by a distance “h” from the center line to the left.

FIG. 13 is a side view in which the LED 51 of the designating tool 1 isviewed from the side. The center line 56 connects a midpoint between theinfrared LEDs 51A and 51C, and the lens 32. FIG. 13 shows a case whenthe designating tool 1 faces a downward diagonal direction at an angle“m” from the center line 56. When the projection plane 54 is assumedalong lines extending perpendicularly from the center line 56, in theprojection plane 54, the space between the infrared LEDs 51A and 51Cbecomes “y”. Moreover, the infrared LED 51E is projected as beingshifted upward by a distance “y” from the center line.

Here, one example of the images 61 which are imaged by the CCD camera31, is shown in FIG. 11. The image 61 can be considered as an image atan arbitrary magnification at the projection plane 54 of FIGS. 12 and13. Therefore, the images 61 of FIG. 11 have the same geometricalrelationships as the projected images in the projection plane 54. InFIG. 11, the images. 61A, 61B, 61C, 61D and 61E are the imagestransmitted by the infrared LEDs 51A, 51B, 51C, 51D and 51E,respectively. Moreover, the space between images 61B and 61D is “X”, thespace between 61A and 61C is “Y”, the intersection of a straight linewhich connects the images 61B and 61D and a straight line which connectsthe images 61A and 61C is a reference point 62, the horizontal componentof the distance between the reference point 62 and the image 61E is “H”,and the vertical component is “V”. The reference point 62 lies on anextension of the center line 56. Therefore, each value of x, h, y and vof FIGS. 12 and 13 becomes proportional to the respective value of X, H,Y and V of FIG. 11. Accordingly, when the relationships of the images61A, 61b, 61C, 61D and 61E are checked, it is possible to find out howmuch the designating tool 1 is inclined horizontally and vertically withrespect to the CCD camera 31.

Accordingly, even when the infrared LEDs 51A, 51B, 51C and 51D aredisposed at the vertices of a rhombus, it can be treated similar to thecase of when the aforementioned infrared LEDs 21A, 21B and 21C aredisposed at the vertices of an isosceles triangle, and the coordinateson the screen can be used by using the same method.

Moreover, in the present embodiment, the case is described where thecenter infrared LED of the designating tool is positioned deeper insidethe designating tool than the surrounding infrared LEDs. Furthermore,another example of this apparatus, wherein the center infrared LEDprotrudes outwardly with respect to the surrounding infrared LEDs, canbe treated the same way by merely reversing the polarity of the outputcoordinates.

In the present embodiment, a method of independently obtaining angles inthe horizontal and vertical planes is shown. However, when both planesare inclined, the angles which appear in the plan and side views areslightly shifted from the accurate values. Moreover, technically, erroroccurs when the designating tool is rotated. However, according to thepresent embodiment, the designating tool is operated while the operatorwatches the marker which is displayed on the screen. Thus, the result ofthe operation is instantly apparent, so that minor errors do notactually cause problems.

A second embodiment of the present invention is explained with referenceto the drawings.

FIG. 14 shows the second embodiment of the present invention. Thedisplay data of the computer 10 is sent to the projector 4, projected,enlarged and displayed on screen 8. The screen 8 is a rear type screen,wherein the side of the screen that is viewed is on the opposite side ofthe screen from the side that is projected onto by the projector 4.

At the top of the screen 8, an imaging part 5 and an illuminating part 6are provided together in a single unit, which irradiates infrared lightto a front surface of the designating tool 2 and images the reflectedimage, detects the orientation of the designating tool 2, converts theorientation into coordinates and transfers the coordinates to thecomputer 10.

During a presentation or the like, the position of the marker 12 can beshifted by the demonstrator 9 by changing the orientation of thedesignating tool 2 while watching the screen.

The marker 12 is the same as a pointer or cursor that is operated by aconventional mouse. The marker 12 can be shifted when the orientation ofthe designating tool 2 is changed, similar to a pointer and cursor beingshifted when the mouse is shifted on a corresponding flat surface.

One example of the designating tool 2 is shown in FIG. 15. A pluralityof reflecting members 71 that reflect infrared rays are provided on thefront face of the designating tool 2.

FIG. 16 is a three-face-view which shows details of the front faceportion of the designating tool 2. The reflecting members 71A, 71B and71C, which are the first reflecting elements, are disposed in the sameplane at the vertices of an isosceles triangle. The reflecting members71A and 71B are separated by a distance “a”. A line that connectsmaterials 71A and 71B is parallel to the top surface of the designatingtool 2. The reflecting member 71C is separated by a distance “b” in thevertical direction from the center of a line which connects reflectingmaterials 71A and 71B. The reflecting member 71E, which is the secondreflecting element, is disposed at a position which is recessed by adistance “d” within the designating tool, separated by a distance b/2 inthe vertical direction from 71C in the front view.

One example of the imaging part 5 and the illuminating part 6 is shownin FIG. 17. The illuminating part 6 has a structure in which manyinfrared LEDs 36 are disposed around the lens 32 of the CCD camera 31,and irradiates in the direction which is imaged by the imaging part 5.Moreover, the imaging part 5 includes the CCD camera 31, the lens 32,the infrared ray filter 33, the image processor 34, and the output part35. The lens 32 and the infrared ray filter 33 are disposed on the CCDcamera 31, and an image of the reflecting member 71 of the designatingtool 2 is imaged. The output of the CCD camera 31 is connected to theimage processor 3,4. The image processor 34 calculates the planarcoordinates on the screen 8 based on the image of the reflecting member71 of the designating tool 2 which has been imaged by the CCD camera 31,and sends it to the computer 10 via the output part 35.

One example of the output image of the CCD camera 31 is shown in FIG. 5.Since the infrared ray filter 33 is disposed in front of the lens 32,the light of interior room illuminations or the like are substantiallyeliminated, as shown in FIG. 5, and only images 41 transmitted by thereflecting member 71 are output.

Here, the transmitting of the images 41 by the designating tool 2 isexplained with reference to the figures.

FIG. 18 is a plan view in which the reflecting member 71 of thedesignating tool 2 is viewed from the top. The center line 76 connects amidpoint between the reflecting members 71A and 71B and the lens 32.FIG. 18 shows the case when the designating tool 2 faces a left diagonaldirection at an angle l from the center line 76. When the projectionplane 74 is assumed on lines extending perpendicularly from the centerline 76, in the projection plane 74, the space between the reflectingmember 71A and 71B becomes “x”, and the reflecting member 71E is shiftedto the left by a distance “h” from the center line and is projected.

FIG. 19 is a side view in which the reflecting member 71 of thedesignating tool 2 is viewed from the side. The center line 76 connectsthe midpoint between the reflecting members 71A, 71B and 71C and thelens 32. FIG. 19 shows a case when the designating tool 2 faces thedownward diagonal direction at an angle m from the center line 76. Whenthe projection plane 74 is assumed on lines extending perpendicularlyfrom the center line 76, in projection plane 74, the distance betweenthe reflecting member 71A, 71B and 71C becomes “y”, and the reflectingmember 71E is shifted upwards by a distance “v” from the center line andis projected.

Here, the images 41 which have been imaged by the CCD camera 31 can beconsidered as images which are imaged at an arbitrary magnification atprojection plane 74 of FIGS. 18 and 19. Thus, the images 41 of FIG. 5have the same geometrical relationship as the projected images in theprojection plane 74. In FIG. 5, the images 41A, 41B, 41C, and 41E arethe images of the reflecting members 71A, 71B, 71C, and 71E,respectively. Furthermore, the distance between the image 41A and 41B isdefined as “X”, the distance between the center of the line connecting41A with 41B and the image 41C in a vertical direction is defined as“Y”, and the position which is above the image 41C by Y/2 is defined asa reference point 42. The horizontal component of the distance betweenthe reference point 42 and the image 41E is defined as “H”, and thevertical component of the distance between the image 41E and thereference point 42 is defined as “V”. Because the reference point 42lies on an extension of the center line 76, each value of x, h, y and vof FIGS. 18 and 19 has a proportional relationship to the respectivevalue of X, H, Y or V of FIG. 5. Therefore, if the relationships of theimages 41A, 41B, 41C, and 41E to the images 41 are checked, it ispossible to determine how much designating tool 2 is inclined in thehorizontal and vertical directions with respect to the CCD camera 31.

[Equation 5]

x=a cos l

h=d sin l${\therefore\frac{h}{x}} = {{{\frac{d}{a}\tan \quad l}\therefore l} = {\tan^{- 1}\left( {\frac{a}{d}\quad \frac{h}{x}} \right)}}$

[Equation 6]

y=b cos m

v=d sin m${\therefore\frac{v}{y}} = {{{\frac{d}{b}\tan \quad m}\therefore\quad m} = {\tan^{- 1}\left( {\frac{b}{d}\quad \frac{v}{y}} \right)}}$

Equation 5 is a formula showing the relationship in the horizontaldirection in the projection plane 74. Furthermore, Equation 6 is aformula showing the relationship in the vertical direction in theprojection plane 74. As described earlier, each value of x, h, y, and vhas a proportional relationship with the respective value of X, H, Y, orV, so Equations 5 and 6 can be defined as Equation 7, which will beshown as follows.

[Equation 7]$l = {\tan^{- 1}\left( {\frac{a}{d}\quad \frac{H}{X}} \right)}$$m = {\tan^{- 1}\left( {\frac{b}{d}\quad \frac{V}{Y}} \right)}$

Here, because the values of a, b, and d of the designating tool 1 arealready-known values, angles l and m can be obtained from the images 41of FIG. 5.

Furthermore, the data which is output to the computer 10 from the outputpart 35 provide horizontally and vertically orthogonal coordinates ofthe plane which is projected onto the screen 8. Therefore, if the centerof the screen is the point of origin, as shown in Equation 8, the anglesl and m can be converted to the coordinates Xs, Ys.

[Equation 8] $\begin{matrix}{{Xs} = {K\quad \tan \quad l}} \\{= {K\quad \frac{a}{d}\quad \frac{H}{X}}}\end{matrix}$ $\begin{matrix}{{Ys} = {K\quad \tan \quad m}} \\{= {K\quad \frac{b}{d}\quad \frac{V}{Y}}}\end{matrix}$

Here, K of Equation 8 is a proportional constant and is a value todetermine sensitivity of output and the inclination of the designatingtool 1. This value can be fixed at an appropriate value which can beeasily used, or can be set in response to the preference of thedemonstrator 9. Furthermore, as understood from Equation 8, the valuesof angles l and m do not need to be obtained in the actual calculation.

Next, according to the above image processor 34, a method of obtainingthe coordinates on the screen which are transmitted by the designatingtool 1 to form the images 41 which are imaged by the CCD camera 31 isexplained by the flow chart of FIG. 8.

In S1, the center of balance coordinates of each of the images 41A, 41B,41C, and 41E are obtained in order to determine a substantially centerposition as a representative point, because the images have a limitedsize. The center position can be obtained by a commonly knowncalculation method.

In S2, the distance X in the horizontal direction and an intermediatecoordinate between the images 41A and 41B are obtained from the centerof balance coordinates of the images 41A and 41B.

In S3, the distance Y in the vertical direction and the coordinates ofthe reference point 42, which is the midpoint of both the distance X andthe distance Y, are obtained from the center of balance coordinates ofthe image 41C and the intermediate coordinate which was obtained in S2.

In S4, the center position of image 41E and the horizontal distance Hand the vertical distance V of the reference point 42 are obtained.

In S5, the coordinates Xs, Ys on the screen 8 are obtained by Equation8.

In summary, a method of obtaining the above coordinates, the distancesX, Y and the reference point 42, are obtained from the images 41A, 41B,and 41C of the reflecting members 71A, 71B, and 71C, which are the firstreflecting elements of the designating tool 2, the horizontal andvertical distances H, V of the reference point 42 are obtained from theimage 41E of the reflecting member 71E, which is the second reflectingelement, and the coordinates Xs, Ys on a screen are obtained by theabove calculation.

Furthermore, the coordinates on the screen 8 use the center as the pointof origin. However, it is possible to use a peripheral part of thescreen as the point of origin by setting a bias value.

The case where only one designating tool 2 is used is explained above.However a method of obtaining the coordinates on the screen designatedby respective designating tools when a plurality of designating toolsare simultaneously used is explained by the flowchart of FIG. 27.

In S1, the image is enlarged and processed for each image. It ispreferable to perform a processing to fill in the periphery of the partin which the image exists by the same signal at a predetermined size.

Next, in S2, the image area is obtained for each designating tool byperforming segmentation.

Since an image area is thus obtained for each designating tool, thecoordinates may be obtained by the flow chart explained in FIG. 8 forthe respective image ranges.

The above invention is explained based upon the embodiment of thedesignating tool 2 shown in FIG. 19. However, as another example of theinvention, it is also possible to perform the same processing as is donewith LEDs by replacing the LEDs 51A, 51B, 51C, 51D, and 51E explained inembodiment 1 with reflecting members.

Thus, even when the reflecting members are disposed at the vertices of arhombus, it is possible to operate in the same manner as the case inwhich the reflecting members 71A, 71B, and 71C are disposed in the shapeof an isosceles triangle. Thus, it is possible to obtain the coordinateson the screen by the same method.

Furthermore, in the present embodiment, the case is explained in whichthe center reflecting member of the designating tool 2 is disposedbehind the surrounding reflecting members. However, as another exampleof the invention, when the center reflecting member protrudes outwardlywith respect to the surrounding reflecting members, it is possible tooperate in the same manner by merely reversing the polarity of theoutput coordinates.

Furthermore, the method of independently obtaining angles in thehorizontal and vertical planes is shown in the present embodiment.However, when both the horizontal and vertical planes are inclined, theangles which will be shown in the plan and side views are technicallydifferent from accurate values. Furthermore, errors also technicallyoccur when the designating tool is rotated. However, in the case of thepresent embodiment, when the designating tool is operated while watchingthe marker which is displayed on the screen, the result of the operationis instantly apparent, so that these errors do not cause a problem.

In addition, a structure in which the imaging part 5 and theilluminating part 6 are integrated is explained in the presentembodiment. However, it is also possible to install them in separatelocations.

The third embodiment of the present invention is explained based uponthe figures.

FIG. 20 shows the third embodiment of the present invention. The displaydata of a computer 10 is sent to a projector 4, and the projected imageis reflected against a mirror 13, is enlarged, and is displayed on ascreen 8. The screen 8 is a rear type screen, wherein the surface thatis observed is on the opposite side as the surface that is projectedonto by the projector 4. Additionally, the projector 4, the mirror 13,and the computer 10 are stored in a case 14, and these elements,collectively, except the computer 10, can be referred to as a rearprojector.

The imaging part 5 and an illuminating part 6 are integrated anddisposed at the bottom of the screen 8. The reflected image is imaged asinfrared light, and is irradiated in front. The direction of thedesignating tool 3 is detected, converted into coordinates, and istransmitted to the computer 10.

During a presentation or the like, the demonstrator 9 can move theposition of the marker 12 by changing the orientation of the designatingtool 3 while watching the screen.

The marker 12 is the same as the pointer or cursor which is operated bya conventional mouse. It is possible to move the marker 12 by changingthe orientation of the designating tool 3 in the same manner that thepointer or cursor is moved by moving the mouse on a corresponding flatsurface.

One example of the designating tool 3 is shown in FIG. 21. A pluralityof reflecting members 81 which reflect infrared radiation, are disposedon the front face of the designating tool.

FIG. 22 is a three-face-view showing the front face of the designatingtool 3 in detail. The reflecting members 81 comprise a reflecting member81A, which is the first reflecting element, and a reflecting member 81B,which is the second reflecting element. The reflecting member 81A is inthe shape of a ring, wherein the center diameter is a distance “a”, andthe reflecting member 81B is disposed in the center, protruding outwardby a distance “d”.

One example of the imaging part 5 and the illuminating part 6 is shownin FIG. 17. The illuminating part 6 has a structure in which manyinfrared LEDs 36 are disposed at the periphery of the lens 32 of the CCDcamera 31, and irradiate in the direction which is imaged by the imagingpart 5. Additionally, the imaging part 5 includes the CCD camera 31, thelens 32, the infrared ray filter 33, the image processor 34, and theoutput part 35. The lens 32 and the infrared ray filter 33 are attachedto the CCD camera 31, and the image of the reflecting members 81 of thedesignating tool 3 is imaged. The output of the CCD camera 31 isconnected to the image processor 34. The image processor 34 calculatesthe planar coordinates on the screen 8, based upon the image of thereflecting members 81 of the designating tool 3 which have been imagedby the CCD camera 31, and outputs them to the computer 10 via the outputpart 35.

One example of an output image of the CCD camera 31 is shown in FIG. 23.Because the infrared ray filter 33 is fixed in front of the lens 32 ofthe CCD camera 31, light such as room illumination can be substantiallyeliminated, and only the image 91 of the reflecting members 81 isoutput, as shown in FIG. 23.

Here, the transmitting of the images 91 by the designating tool 3 isexplained by the diagrams.

FIG. 24 is a plan view of the reflecting members 81, as seen from thetop by cutting the designating tool 3 in a horizontal plane through itscenter. The center line 86 is a line connecting the center of thereflecting member 81A with the lens 32. FIG. 24 shows the case when thedesignating tool 3 is inclined from the center line 86 at an angle l inthe left diagonal direction. If the projection plane 84 is assumed alonglines extending perpendicularly from the center line 86, the interval ofthe reflecting member 81A in the projection plane 84 in the diameterdirection is “x”, and the reflecting member 81B is shifted to the leftfrom the center line by a distance “h” and s projected.

FIG. 25 is a side view of the reflecting members 81 by cutting thedesignating tool 3 along a vertical plane through its center. The centerline 86 is a line connecting the center of the reflecting member 81Awith the lens 32. FIG. 25 shows the case when the designating tool 3faces a downward diagonal direction from the center line 86 at an anglem. If the projection plane 84 is assumed along lines extendingperpendicularly from the center line 86, the interval of the reflectingmember 81A in the diameter direction is “y” in the projection plane 84,and the reflecting member 81B is shifted in an upper direction by adistance “y” from the center line and is projected.

Here, the images 91 which are imaged by the CCD camera 31 can beconsidered as images which are imaged at an arbitrary magnification atthe projection plane 84 of FIGS. 24 and 25. So the images 91 of FIGS. 23have the same geometrical relationship as the projected images in theprojection plane 84. In FIG. 23, the images 91A and 91B are images ofthe reflecting members 81A and 81B, respectively. Furthermore, themaximum interval of the image 91A in the horizontal direction is definedas “X”, the maximum interval in the vertical direction is defined as“Y”, and the crossing point of vertical and horizontal lines whichtravel through the center of the respective intervals is defined as thereference point 92. Furthermore, the horizontal component of thedistance between the reference point 92 and the image 91B is defined as“H”, and the vertical component is defined as “V”. Because the referencepoint 92 lies on an extension of the center line 86, each value of x, h,y, and v of FIGS. 24 and 25 has a substantially proportionalrelationship to the respective value of X, H, Y, or V of FIG. 23.Accordingly, if the relationships of images 91A and 91B of the images 91are checked, it is possible to determine how much the designating toolis inclined in the horizontal and vertical directions with respect tothe CCD camera 31.

[Equation 9]

x=a cos l

h=d sin l${\therefore\frac{h}{x}} = {{{\frac{d}{a}\tan \quad l}\therefore l} = {\tan^{- 1}\left( {\frac{a}{d}\quad \frac{h}{x}} \right)}}$

[Equation 10]

y=a cos m

v=d sin m${\therefore\frac{v}{y}} = {{{\frac{d}{b}\tan \quad m}\therefore\quad m} = {\tan^{- 1}\left( {\frac{b}{d}\quad \frac{v}{y}} \right)}}$

Equation 9 is a formula showing the relationship in the horizontaldirection of the projection plane 84. Furthermore, Equation 10 is aformula showing the relationship in the vertical direction of theprojection plane 84. As described earlier, each value of x, h, y, and vhas a substantially proportional relationship to the respective value ofX, H, Y, or V of FIG. 23, so Equations 9 and 10 can be defined asEquation 11 which will be shown below.

[Equation 11]$l = {\tan^{- 1}\left( {\frac{a}{d}\frac{H}{X}} \right)}$$m = {\tan^{- 1}\left( {\frac{a}{d}\frac{V}{Y}} \right)}$

Here, the values of a, d are already-known values of the designatingtool 3, so the angles l and m can be obtained from the images 91 of FIG.23.

Furthermore, the data which is output to the computer 10 from the outputpart 35 provides horizontally and vertically orthogonal coordinates ofthe plane which is projected onto the screen 8. Therefore, if the centerof the screen is the point of origin, it is possible to convert theangles l and m to the coordinates Xs, Ys, as shown in Equation 12.

[Equation 12] $\begin{matrix}{{Xs} = {K\quad \tan \quad l}} \\{= {K\frac{a}{d}\frac{H}{X}}}\end{matrix}$ $\begin{matrix}{{Ys} = {K\quad \tan \quad m}} \\{= {K\frac{a}{d}\frac{V}{Y}}}\end{matrix}$

Here, K of Equation 12 is a proportional constant and is a value todetermine the sensitivity of output and the inclination of thedesignating tool 3. This value can be fixed at an appropriate valuewhich can be easily used, or can be set in response to the preference ofthe demonstrator 9. Furthermore, as understood from Equation 12, thevalues of angles l, m do not need to be obtained in the actualcalculation.

Next, in accordance with the image processor 34, a method of obtainingcoordinates on the screen which are designated by the designating tool 3from the images 91 which have been imaged by the CCD camera 31 isexplained by the flowchart of FIG. 26.

In S1, the positions of the right and left edges of the image 91A in thehorizontal direction are obtained, the center of balance coordinates inthe horizontal direction of each edge part are obtained, and thedistance X between both edge parts in the horizontal direction isobtained.

In S2, the positions of the upper and lower edges of the image 91A inthe vertical direction are obtained, the center of balance coordinatesin the vertical direction of each edge part are obtained, and thedistance Y between both edge parts in the vertical direction isobtained.

In S3, the coordinates of the reference point 92, which is at the centerposition of both X and Y, respectively, are obtained.

In S4, the center of balance coordinate of the image 91B is obtained,and the distance between the image 91B and the coordinates of thereference point 92 is obtained as H in the horizontal direction and as Vin the vertical direction.

In S5, the coordinates Xs, Ys on the screen are obtained by Equation 12.

In summary, in accordance with the method of obtaining the abovecoordinates, the distances X, Y and the reference point are obtainedfrom the image 91A of the reflecting member 81A, which is the firstreflecting element of the designating tool 3, the horizontal andvertical distances H, V between the reference point and the image 91B ofthe reflecting member 91B, which is the second reflecting element, areobtained, and the coordinates Xs, Ys on the screen are obtained by acalculation.

Furthermore, the coordinates on the screen use the center as the pointof origin, but it is possible to use the periphery of the screen as thepoint of origin by setting a bias value.

The case in which only one designating tool 3 is used has been explainedabove. However, a method for obtaining the coordinates on the screendesignated by respective designating tools when a plurality ofdesignating tools are simultaneously used is explained by the flowchartof FIG. 27.

In S1, the image is enlarged and processed for each image. It ispreferable to fill in the periphery of the part in which the imageexists by the same signal at a predetermined size.

Next, in S2, the image area is obtained for each designating tool byperforming segmentation.

Because the image area is thus obtained for the designating tool, thecoordinates can be obtained by the flow chart explained in FIG. 26 withregard to the respective image areas.

As explained above, in the present embodiment, the case in which thereflecting member at the center of the designating tool 3 protrudesoutwardly with respect to the reflecting member at the periphery isexplained. However, as another example of the invention, when thereflecting member at the center is positioned behind the reflectingmember at the periphery, it is possible to operate in the same manner bymerely reversing the polarity of the output coordinates.

Furthermore, a method for obtaining angles independently in thehorizontal and vertical planes is shown in the present embodiment, andwhen both planes are inclined, the angle which appears in the plan andside views is technically not accurate. However, as in the presentembodiment, when the demonstrator operates the designating tool whilewatching the marker which is displayed on the screen, the operationresult is instantly apparent, so that minor errors do not cause aproblem.

As explained above, according to the present invention, the designatingtool includes at least three light emitting elements and a lightemitting element which is located on an orthogonal axis to a planedelineated by the at least three light emitting elements, the relativepositional relationship of the light emitting elements is imaged by animage pick-up means, and the orientation in which the designating toolpoints toward the image pick-up means is obtained from the image signaland is converted to planar coordinates. Therefore, it is possible to useapplication software of the computer by moving the marker on the screenas the demonstrator moves the designating tool. Because of this, it ispossible for the demonstrator to explain items by pointing them out, asif with a laser pointer, without having to move from in front of thescreen during a presentation where a computer screen is enlarged andprojected by a projector. Therefore, it is possible to prevent theaudience from being distracted.

Furthermore, light emitting elements are disposed at each vertex of anisosceles triangle, and the base of the triangle connecting two of thelight emitting elements is substantially horizontally disposed, sosquare and/or root calculations are not needed in order to obtain thediagonal distance and coordinates. Therefore, it is possible to simplifythe calculation to obtain the coordinates from the image which has beenimaged.

Furthermore, light emitting elements are disposed at each vertex of arhombus, and one diagonal line of the diagonal shape is substantiallyhorizontally disposed, so the square and/or root calculations are notneeded in order to obtain the diagonal distance and coordinates. Thus,it is possible to simplify the calculation to obtain the coordinatesfrom the image which has been imaged.

Furthermore, a modulating means to modulate the light emitting elementsin accordance with the operating information from the operating meansand a light receiving means to detect the modulated operatinginformation are provided, so it is possible to operate applicationsoftware the same way as is done with a mouse, and operations such aspage scrolling or the like can be performed, using only the designatingtool, during the presentation, and distraction of the flow of thepresentation can be avoided.

In addition, a hand detecting means is provided on the designating tool,and turning on and off of the light emitting elements can be controlled,so that it is possible to turn the lights off when it is not being used,and turn the lights on when it is being used. Thus, it is possible toavoid wasting energy, such as batteries, and to have a long lifeexpectancy of the light emitting elements.

Furthermore, the designating tool includes at least three reflectingelements and a reflecting element which is disposed on an orthogonalaxis to a plane delineated by the at least three light emittingelements, the relative positional relationship of the reflectingelements irradiated by an irradiating means is imaged by an imagepick-up means, and the direction in which the designating tool pointswith respect to the image pick-up means is obtained from the imagesignal and is converted to planar coordinates. Therefore, it is possibleto use application software of a computer by moving a marker on thescreen as the demonstrator moves the designating tool. Because of this,it is possible for the demonstrator to explain items by pointing themout, as if with a laser pointer, without having to leave the view of theaudience during a presentation performed by enlarging the image onto acomputer screen and projecting it with a projector. Thus, it is possibleto prevent the audience from being distracted. Furthermore, thedesignating tool does not need an energy source, such as a battery, somaintenance management can be easily performed.

Furthermore, the reflecting elements are disposed at each vertex of anisosceles triangle, and the base of the triangle connecting two of thereflecting elements is substantially horizontally disposed, so squareand/or root calculations are not needed in order to obtain the diagonaldistance and coordinates. Thus, it is possible to simplify thecalculation in order to obtain the coordinates from the image which hasbeen imaged.

In addition, the reflecting elements are disposed at each vertex of arhombus, and one diagonal line of the rhombus is substantiallyhorizontally disposed, so square and/or root calculations are not neededin order to obtain the diagonal distance and coordinates. Thus, it ispossible to simplify the calculation to obtain the coordinates from theimage which has been imaged.

Furthermore, the designating tool includes a reflecting element of ahollow disk shape which is disposed in a single plane, and a reflectingelement which is disposed on an orthogonal axis to the plane delineatedby the disk-shaped reflecting element, the relative positionalrelationship of the reflecting devices which have been irradiated by anirradiating means is imaged by an imaging means, and the direction inwhich the designating tool points with respect to the imaging means isobtained from the image signal and is converted to planar coordinates.Therefore, it is possible to use the application software of thecomputer by moving the marker on the screen as the demonstrator movesthe designating tool. Because of this, it is possible for thedemonstrator to explain items by pointing them out, as if with a laserpointer, without having to leave the view of the audience during apresentation or the like in which an image of a computer screen isenlarged and projected by the projector. Therefore, it is possible toprevent the audience from being distracted. Furthermore, the designatingtool does not need an energy source, such as a battery, so themaintenance management can be easily performed. In addition, thereflecting element of the designating tool is radially symmetrical, soit is possible to use the designating tool regardless of the manner inwhich the demonstrator holds the designating tool.

Furthermore, in a method of the invention, a first image from the firstlight emitting element or the first reflecting element and a secondimage from the second light emitting element or the second reflectingelement are obtained from an image which has been imaged by the imagingmeans, reference coordinates are obtained from the coordinates of thefirst image, the orientation of the designating tool with respect to theimaging means is obtained from the positional relationship of thereference coordinates and the second image, and the designating positionon the display means is specified according to the orientation.Therefore, the size of the image which has been imaged is not limited.Even if the distance between the imaging means and the designating toolis not constant, it is possible to specify the designating position onthe screen and the demonstrator can move the position. Furthermore, thescreen corner of the image pick-up means can be arbitrarily set, so theattachment location of the imaging means is not limited, and adjustmentscan be easily performed.

In addition, a method is also provided wherein independent designatingtool images which are separated from an image of a plurality designatingmeans into independent images are obtained from an image which wasimaged by the imaging means, a plurality of images of designating meanscan be obtained from the independent designating tool images which weredivided into independent images, the orientation of each designatingmeans relative to the imaging means is obtained for each independentdesignating tool image, and the designating position on said displaymeans is specified according to the orientation. Therefore, it ispossible to simultaneously point out two or more locations, andsimultaneously perform input at two locations on the screen, such as oneselected by the demonstrator and one selected by a member of theaudience of the presentation.

What is claimed is:
 1. A remote coordinate input system for use with ascreen, comprising: a designator including a first surface on which atleast three first reflecting elements are arranged in a plane, and asecond reflecting element arranged on an orthogonal axis to the plane,said second reflecting element arranged on a second surface differentfrom said first surface; an imaging device that images relativepositional relationship between the first reflecting elements and thesecond reflecting element; an irradiating device that irradiates the atleast three first reflecting elements and the second reflecting element;a coordinate converter that obtains an orientation of the designatorwith respect to said imaging device from an image that is imaged by saidimaging device and converts the image into planar coordinates; an outputdevice that outputs the planar coordinates that are obtained by thecoordinate converter; and a display that displays designationinformation on the screen based on the planar coordinates that areobtained from said output device.
 2. The remote coordinate input systemof claim 1, each of the at least three first reflecting elements beingrespectively disposed at each vertex of an isosceles triangle, and abase of the isosceles triangle that is formed by connecting two of theat least three reflecting elements is substantially horizontallyarranged.
 3. The remote coordinate input system of claim 1, each of theat least three first reflecting elements being respectively arranged ateach vertex of a rhombus, one diagonal line of said rhombus beingsubstantially horizontally arranged.
 4. A remote coordinate input systemfor use with a screen, comprising: a designator including a firstsurface on which a hollow disk-shaped first light reflecting element isarranged on a plane and a second light reflecting element arranged on anorthogonal axis to the plane said second reflecting element arranged ona second surface on a plane different from said first surface; animaging device that images relative positional relationship between thehollow disk-shaped first light reflecting element and the second lightreflecting element; an irradiating device that irradiates the hollowdisk-shaped first light reflecting element and the second lightreflecting element; a coordinate converter that obtains an orientationof the designator with respect to said imaging device from an imagewhich is imaged by said imaging device and converts the image to planarcoordinates; an output device that outputs the planar coordinates thatare obtained by the coordinate converter; and a display that displaysdesignating information on the screen based on the planar coordinatesthat are obtained from said output device.
 5. A remote coordinate inputmethod, comprising the steps of: obtaining a first image from a firstreflecting element; obtaining a second image from a second reflectingelement; obtaining a reference coordinate from a coordinate of saidfirst image; obtaining an orientation of a designator with respect to animaging device from a positional relationship between the second imageand the reference coordinate; and specifying a designating position on adisplay according to said orientation.
 6. The remote coordinate inputmethod of claim 5, further comprising the steps of: obtainingindependent designating tool images in which an image of a plurality ofdesignators are separated into independent images; obtaining anorientation of each designator with respect to the imaging device foreach independent designating tool image; and specifying designatedpositions on the display according to said orientations.